Repository navigation
fix(compose): forward batch tokens and strict Jev knobs to laya-serve - #1054
Conversation
|
Merged. Same shape as #1053 and the same real gap:
Merge note: this and #1053 both extend the same name tuple in |
14 pull requests from 8 contributors. * the TypeScript tree parses again: an unterminated template literal shipped in 0.4.1 and stopped 22 of 34 test files loading (#1056) * x_jev_confidence on /v1/systemone, computed locally (#1066, #302) * a narrowed shortlist drops option_order with a warning instead of failing (#1063) * registering a name one edit from a built-in warns (#1055, #919) * compose forwards four knobs the runtime already read (#1053, #1054) * Java Router.predictBatch and Shortlist.predictTournament (#1060, #1061) * teacher-agreement evals, with the self-agreement ceiling (#1071, #919) Security: no malicious content found; the audit is recorded locally. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What
LAYA_MAX_BATCH_TOKENSandLAYA_JEV_STRICTfrom host environment variables throughcompose.http.yamlto thelaya-servecontainer.docs/docker.md'sServer configurationtable andlaya/serve.pymodule docstring.tests/test_compose_env.py.laya-ts/src/hooks.tsandlaya-ts/src/agent.tsto keep the TypeScript CI suite green.Why
laya.servereads two runtime knobs for batch inference and client compatibility:LAYA_MAX_BATCH_TOKENS: budgets tokens per forward pass forPOST /v1/systemone/batch(DEFAULT_MAX_BATCH_TOKENS = 131072), chunking requests that exceed it to protect workers against OOMs.LAYA_JEV_STRICT: projects responses onto the strict Jev wire contract (stripping extra metadata fields that cause validation failures on strict Jev clients).Both are documented in
docs/http-api.mdand wired intonix/laya-serve.nix, but were omitted fromcompose.http.yamlanddocs/docker.md. When an operator configured them for a Docker Compose deployment, Compose silently dropped the variables because they were not declared underservices.laya-serve.environment.This achieves 100% reachability and parity across all server runtime configuration dials in Compose.
How it was verified
Executed the test suites locally with zero errors:
Checklist
main